1. Field of the Invention
The present invention relates to an image coded data re-encoding apparatus for producing a second image coded data by applying digital signal processing to a first image coded data which is obtained by coding a digital input image signal.
2. Description of Related Art
FIG. 16 is a block diagram showing a conventional image coded data re-encoding apparatus disclosed in Japanese patent application laid-open No. 2-179186/1990, for example. It is an example of a multiple site video conference system. In this figure, the reference numeral 1 designates a master station, 2 designates a relay station, 3 designates a slave station, 10 designates an image coder of the stations, and 20 designates an image decoder. The reference numerals 101 designates an input image, 102 designates image coded data, 103 designates a decoded image signal, 104 designates image re-encoded data, and 105 designates a decoded image signal.
The operation will now be described.
The image coder 10 of the master station 1 performs coding of the input image 101, and sends the image coded data 102 to the relay station 2. The relay station 2 receives the image coded data 102 with its image decoder 20 to decode it, and generates the image re-encoded data 104 by re-encoding the decoded image signal 103 with the image coder 10. The image re-encoded data 104 thus generated by the re-encoding is transmitted to the slave station 3. The slave station 3 decodes with the image decoder 20 the image re-encoded data 104 which is relayed through the relay station 2, and uses it as the decoded image signal 105.
When holding a conference using the relay station 2 with such a decoding and relaying function, it often occurs that the master station 1 and the slave station 3 employ different coding systems. In this case, it becomes necessary to change the amount of coded data to be generated and various types of parameters such as image size and a frame rate. Thus, the relay station 2 once decodes the received image coded data 102 into the decoded image signal 103, and then re-encodes it into the image re-encoded data 104, thereby matching the different coding systems.
In this way, the conventional image coded data re-encoding apparatus has a process through which the image coded data is once decoded to be re-encoded to achieve relaying or copying of the image coded data.
Since the conventional image coded data re-encoding apparatus with such an arrangement once decodes the image coded data 102 into the decoded image signal 103, and then re-encodes the decoded image signal 103 regardless of its contents to relay or convert the image coded data 102, it has some problems such as degrading the image quality of the decoded image signal 105, increasing a delay involved in the relay and transform, and augmenting the size of the apparatus.
The present invention is accomplished to solve such problems, and to provide an image coded data re-encoding apparatus which can reduce the image degradation, shorten the processing delay, and shrink the device size, thereby achieving an efficient transform of the image coded data.
According to one aspect of the present invention, there is provided an image coded data re-encoding apparatus comprising: an image coded data analyzer for generating coded data after signal processing by performing a first digital signal processing on a first image coded data; and an image coded data synthesizer for generating a second image coded data by performing on the coded data after signal processing a second digital signal processing based on multiple signals associated with a first image coded data by using the coded data after signal processing output from the image coded data analyzer and the multiple signals.
This will offer an advantage of reducing the degradation of the image quality after the transform, decreasing processing delay, and achieving an image coded data re-encoding apparatus with a reduced size in comparison with a system in which the first image coded data is always once decoded into a decoded image, followed by re-encoding of the decoded image into the second image coded data regardless of the decoded data.
In image coded data re-encoding apparatus, the image coded data analyzer may extract the multiple signals in the course of generating the coded data after signal processing by performing the first digital signal processing on the first image coded data, and may provides the image coded data synthesizer with the coded data after signal processing.
This will offer an advantage of providing an information effective image coded data re-encoding apparatus capable of obviating special additional information which was needed for the second digital signal processing for generating the second image coded data.
In the image coded data re-encoding apparatus, the image coded data re-encoding apparatus may further comprise a separator for separating from the first image coded data the multiple signals which have been externally combined with the first image coded data and cannot be extracted from the first image coded data in the first digital signal processing for generating the coded data after signal processing, and the image coded data synthesizer may generate the second image coded data by using the multiple signals output from the separator.
This will offer an advantage of providing a coded data re-encoding apparatus which can use, in the second digital signal processing for generating the second image coded data, information which cannot be extracted in course of the first digital signal processing, and this will make possible more effective transform than when such information is not used.
The image coded data re-encoding apparatus may further comprise an information extractor/estimator for extracting or estimating the multiple signals needed for the second digital signal processing from the coded data after signal processing generated by the image coded data analyzer, and the image coded data synthesizer may generate the second image coded data by using the multiple signals output from the information extractor/estimator.
This will offer an advantage of providing an image coded data re-encoding apparatus with a simple configuration because it obviates special processing involved in decoding.
In the image coded data re-encoding apparatus, the image coded data synthesizer may generate the second image coded data with a data amount different from a data amount of the first image coded data input to the image coded data analyzer.
This will offer an advantage of providing an image coded data re-encoding apparatus which can perform transformation between data of different amounts with reduced image degradation and processing delay.
The image coded data re-encoding apparatus may further comprise a coefficient deletion/addition/correction portion for deleting part of the transform coefficients or quantization indices, which are extracted by the image coded data analyzer, and for correcting the transform coefficients or quantization indices in accordance with a ratio of amounts of data to be transformed.
This will offer an advantage that the amount of data can be reduced because part of the transform coefficients or the quantization indices is deleted, and that the image quality of the decoded image can be improved as compared with a system which simply thins out the transform coefficients or quantization indices because the remainder of the transform coefficients or the quantization indices is corrected in accordance with the ratio of amounts of data to be transformed.
The image coded data re-encoding apparatus may further comprise a coefficient deletion/addition/correction portion for deleting part of the transform coefficients or quantization indices, which are extracted by the image coded data analyzer, and are weighted in accordance with relationships between the transform coefficients or quantization indices and their neighboring transform coefficients or quantization indices.
This will offer an advantage that the amount of data can be reduced, and the image quality of the decoded image can be improved as compared with a system which simply thins out the transform coefficients or quantization indices.
The image coded data re-encoding apparatus may further comprise a coefficient deletion/addition/correction portion for adding, to the transform coefficients or quantization indices which are extracted in the image coded data analyzer, new transform coefficients or quantization indices after correcting the new transform coefficients or quantization indices in accordance with a ratio of amounts of data to be transformed.
This will offer an advantage of achieving transform with improved image quality as compared with a system which simply adds the transform coefficients or quantization indices because the newly added transform coefficients or quantization indices are corrected in accordance with the ratio of amounts of data to be transformed, and hence the data amount of the additional data can be adjusted considering the image quality after inverse transform.
The image coded data re-encoding apparatus may further comprise a coefficient deletion/addition/correction portion for adding, to the transform coefficients or quantization indices which are extracted in the image coded data analyzer, new transform coefficients or quantization indices after predicting transform coefficients or quantization indices including the new transform coefficients or quantization indices and their neighboring transform coefficients or quantization indices.
This will offer an advantage of achieving transform with improved image quality, providing clearer images than a system which simply adds the transform coefficients or quantization indices, because such prediction is carried out as improving the decoded image quality in adding the transform coefficients or quantization indices.
The image coded data re-encoding apparatus may further comprise a coefficient deletion/addition/correction portion for increasing a ratio of deletion of the transform coefficients or quantization indices, which are extracted by the image coded data analyzer, if the coding parameter designating the picture type indicates, when a decision is made whether or not the current image to be processed is used for prediction in future coding, that the picture type is not used for the prediction in the future coding.
This will offer an advantage of achieving high quality transform which can maintain the total image quality on the time axis because when a frame of unit time length is not used for the coding in the next unit time, only the data amount associated with the frame can be reduced.
The image coded data re-encoding apparatus may further comprise a coefficient deletion/addition/correction portion for decreasing a ratio of deletion of the transform coefficients or quantization indices, which are extracted by the image coded data analyzer, if the coding parameter designating the picture type indicates, when a decision is made whether or not the current image to be processed is used for prediction in future coding, that the picture type is used for the prediction in the future coding.
This will offer an advantage of achieving high quality transform which can maintain the total image quality on the time axis because when a frame of unit time length is used for the coding in the next unit time, the reduction ratio of the data in the frame is decreased.
The image coded data re-encoding apparatus may further comprise a coefficient deletion/addition/correction portion for increasing a ratio of deletion of the transform coefficients or quantization indices, which are extracted by the image coded data analyzer, if the coding parameter designating the predictive type of the image block indicates, when a decision is made whether a current image to be processed is used for prediction in future coding by using the coding parameters generated by the image coded data analyzer, that the image block is not used for the prediction in the future coding, even if the coding parameter designating the picture type indicates that the picture type is used for the prediction in the future coding.
This will offer an advantage of achieving transform with improved image quality on the time axis because the decision is made of an increase of the deletion ratio of the transform coefficients or the quantization indices by using the coded block information in addition to the coded picture information, and hence finer control becomes possible.
In the image coded data re-encoding apparatus, the image coded data synthesizer may generate the second image coded data whose decoding procedure differs from a decoding procedure of the first image coded data input to the image coded data analyzer.
This will offer an advantage of providing higher quality image after the transform when generating image coded data between different coding systems.
The image coded data re-encoding apparatus may further comprise a coding parameter corrector/transformer for correcting and transforming the expression form of various types of coding parameters, which are extracted by the image coded data analyzer, from the expression form in the decoding procedure of the first image coded data to an expression form in the decoding procedure of the second image coded data.
This will offer an advantage of providing a small size, inexpensive apparatus because it becomes unnecessary for the first image coded data to be once decoded into an image and then re-encoded in accordance with the coding system, even when the first image coded data input to the image coded data analyzer has a different coding system from the second coded data output from the image coded data synthesizer.
In the image coded data re-encoding apparatus, the image coded data synthesizer may generate the second image coded data including an image signal whose image size differs in time or space from an image size of an image signal included in the first image coded data input to the image coded data analyzer.
This will offer an advantage of providing an image coded data re-encoding apparatus which facilitates the transform between the different image sizes in time or space, thereby achieving high quality image after the transform.
The image coded data re-encoding apparatus, may further comprise a coefficient deletion/addition/correction portion for changing an amount of the transform coefficients or quantization indices extracted by the image coded data analyzer, and for correcting the transform coefficients or quantization indices, which are extracted by the image coded data analyzer, in accordance with a ratio of the image sizes to be transformed.
This will offer an advantage of reducing the degradation of the image quality after the change in the image size by suppressing sharp degradation in the resolution or unnaturalness of the image, because the transform coefficients or the quantization indices are corrected in accordance with the image sizes to be varied in the coded data transform involving the image size change.
The image coded data re-encoding apparatus may further comprise a coefficient deletion/addition/correction portion for correcting dimension of the motion vectors extracted by the image coded data analyzer in accordance with a ratio of the image sizes to be transformed.
This will offer an advantage that a characteristic is obtained, which substantially matches the characteristic obtained in a wider motion compensative search range, by a narrower range motion compensative search, because of the improved motion compensation efficiency in the image coded data after the transform since the dimension of the motion vectors are corrected in accordance with the ratio of the image sizes to be changed.
In the image coded data re-encoding apparatus, the image coded data synthesizer may generates the second image coded data including an image signal whose sequence differs from a sequence of an image signal included in the first image coded data input to the image coded data analyzer.
This will offer an advantage of providing an image coded data re-encoding apparatus that can achieve transform between image coded data whose image signal sequences are different, thereby achieving high image quality after the transform.
The image coded data re-encoding apparatus may further comprise a motion searcher for estimating dimension of the motion vectors extracted by the image coded data analyzer in accordance with the sequence of the image signals to be transformed.
This will offer an advantage of improving the efficiency of coding using the motion vectors after the transform by estimating the dimension of the motion vectors in accordance with the sequences of the images to be transformed.
In the image coded data re-encoding apparatus, the image coded data synthesizer may generate the second image coded data whose decoded image signal includes a number of frames per unit time different from a number of frames per unit time of a decoded image signal of the first image coded data input to the image coded data analyzer.
This will offer an advantage of providing an image coded data re-encoding apparatus capable of implementing the transform into the image coded data whose number of image frames differs from that of the decoded image signal, thereby achieving high image quality after the transform.
The image coded data re-encoding apparatus may further comprise a quantization estimator for estimating, from the decoded image output from the image coded data analyzer, quantization parameters obtained in the course of generating the first image coded data, and the image coded data synthesizer may generate the second image coded data by using the quantization parameters estimated by the quantization estimator.
This is effective when the image coded data analyzer carries out operation matching a common decoding operation, and will offer an advantage of providing an image coded data re-encoding apparatus capable of maintaining the image quality after the transform in such application as relay transmission because the estimation of the quantization indices from the decoded image makes it possible to improve the image quality of the second image coded data. In addition, it offers an advantage of implementing an optimum transform by controlling the quantization parameters in accordance with the ratio of data rates before and after the transform. In particular, the best image quality can be achieved when the rates are identical before and after the transform.